<!DOCTYPE html>
<html>

<head>
  <meta charset="UTF-8">
  <script src="../node_modules/@bower_components/webcomponentsjs/webcomponents-lite.min.js"></script>
  <link rel="import" href="ganlab.html">

  <link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/dialog-polyfill/0.4.9/dialog-polyfill.min.css" />
  <script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/dialog-polyfill/0.4.9/dialog-polyfill.min.js"></script>
  <script src="../lib/support.js"></script>
  
  <title>GAN Lab: Play with Generative Adversarial Networks in Your Browser!</title>
  <style>
    html {
      width: 100%;
      height: 100%;
    }

    body {
      font-family: "Roboto", "Helvetica", "Arial", sans-serif;
      margin: 0;
      min-width: 1442px;
      width: 100%;
      height: 100%;
      --discriminator-color: rgb(68, 124, 228);
      --generator-color: rgb(163, 62, 189);
      --gradient-color: rgb(196, 20, 219);
      --real-data-color: rgb(0, 136, 55);
    }
  
    #top-bar {
      background-color: #183D4E;
      color: #f5f5f5;
      font-size: 16px;
      padding: 7px 0 6px;
    }

    #top-bar-panel-container {
      align-items: center;
      display: flex;
      margin-left: auto;
      margin-right: auto;
      justify-content: space-between;
      width: 1442px;
    }

    @media screen and (min-device-width: 1024px) and (max-width: 1442px) {
      #top-bar-panel-container {
        margin-left: 0;
        width: 98.5vw;
      }
    }

    #tool-description {
      padding-left: 20px;
    }

    #source-link {
      padding-right: 20px;
    }

    #source-link a {
      color: #f0f0f0;
      font-size: 14px;
      text-decoration: none;
    }

    #loading {
      background-color: #f7f7f7;
      font-size: 40px;
      height: 140px;
      line-height: 40px;
      padding-top: 60px;
      position: absolute;
      text-align: center;
      width: 100%;
      z-index: -1;
    }

    #loading small {
      font-size: 20px;
    }

    #demo {      
      min-height: 200px;
      width: 100%;
    }

    #description {
      margin-bottom: 60px;
      margin-left: auto;
      margin-right: auto;
      max-width: 660px;
    }

    #description h2 {
      color: #444;
      font-size: 32px;
      font-weight: 450;
      margin-bottom: 12px;
      margin-top: 60px;
    }

    #description h4 {
      color: #444;
      font-size: 24px;
      font-weight: 450;
      margin-bottom: 8px;
      margin-top: 44px;
    }

    #description p {
      margin: 16px 0;
    }

    #description p, 
    #description div,
    #description li,
    #description dt,
    #description dd {
      color: #555;
      font-size: 17px;
      line-height: 1.6;
    }
      
    #description .supporting-figure {
      margin-bottom: 35px;
      margin-top: 30px;
    }
    
    #description .figure-caption {
      font-size: 14px;
      margin-top: 5px;
    }
      
    #description dt {
      display: list-item;
      list-style-type: disc;
      margin-left: 30px;
    }

    #description ul {
      margin-top: -10px;
    }

    #description a, 
    #description .video-part-link {
      color: #0D658C;
      cursor: pointer;
      font-weight: normal;
      text-decoration: none;
    }
      
    #description a:hover, 
    #description .video-part-link:hover {
      text-decoration: underline;
    }

    #description div.citation {
      font-size: 14px;
      line-height: 20px;
      margin-top: 0px;
      padding: 20px 10px 0 110px;
    }
  </style>
</head>

<body>
  <div id="top-bar">
    <div id="top-bar-panel-container" class="panel-container">
      <div id="tool-description">
        Play with Generative Adversarial Networks (GANs) in your browser!          
      </div>
      <div id="source-link">
        <a href="https://github.com/poloclub/ganlab" title="Source on GitHub">
          Fork us on GitHub&nbsp;
          <img src="images/github-mark.png" />
        </a>
      </div>
    </div>
  </div>      
      
  <div id="demo">
    <div id="loading">
      Loading...
      <br />
      <small>It may take a few tens of seconds.</small>
    </div>
    <gan-lab></gan-lab>
  </div>

  <div id="description">
    <h2>What is a GAN?</h2>
    <p>
      Many machine learning systems look at some kind of complicated input (say, an image) and produce a simple output (a label like, "cat"). By contrast, the goal of a generative model is something like the opposite: take a small piece of input&mdash;perhaps a few random numbers&mdash;and produce a complex output, like an image of a realistic-looking face. A <strong>generative adversarial network (GAN)</strong> is an especially effective type of generative model, <a href="https://arxiv.org/pdf/1406.2661.pdf" title="The original paper by Goodfellow et al. published in 2014">introduced only a few years ago</a>, which has been a subject of intense interest in the machine learning community. 
    </p>
    
    <p>
      You might wonder why we want a system that produces realistic images, or plausible simulations of any other kind of data. Besides the intrinsic intellectual challenge, this turns out to be a surprisingly handy tool, with <a href="https://arxiv.org/pdf/1701.00160.pdf" title="See page 4 of this tutorial paper">applications ranging from art to enhancing blurry images</a>.
    </p>      

    <h2>How does a GAN work?</h2>
    <p>
      The idea of a machine "creating" realistic images from scratch can seem like magic, but GANs use <em>two key tricks</em> to turn a vague, seemingly impossible goal into reality.
    </p>
    
    <p>
      The first idea, not new to GANs, is to use <em>randomness</em> as an ingredient. At a basic level, this makes sense: it wouldn't be very exciting if you built a system that produced the same face each time it ran. Just as important, though, is that thinking in terms of probabilities also helps us translate the problem of generating images into a natural mathematical framework. We obviously don't want to pick images at uniformly at random, since that would just produce noise. Instead, we want our system to learn about which images are likely to be faces, and which aren't. Mathematically, this involves modeling a probability distribution on images, that is, a function that tells us which images are likely to be faces and which aren't. This type of problem&mdash;modeling a function on a high-dimensional space&mdash;is exactly the sort of thing neural networks are made for.
    </p>
    
    <p>      
      The big insights that defines a GAN is to set up this modeling problem as a kind of contest. This is where the "adversarial" part of the name comes from. The key idea is to build not one, but two competing networks: a <span style="color: var(--generator-color);"><strong>generator</strong></span> and a <span style="color: var(--discriminator-color);"><strong>discriminator</strong></span>. The generator tries to create random <span style="color: var(--generator-color);">synthetic outputs</span> (for instance, images of faces), while the discriminator tries to tell these apart from <span style="color: var(--real-data-color);">real outputs</span> (say, a database of celebrities). The hope is that as the two networks face off, they'll both get better and better&mdash;with the end result being a generator network that produces realistic outputs.
    </p>
    
    <p>      
      To sum up: <strong><em>Generative adversarial networks</em></strong> are neural networks that learn to choose samples from a special distribution (the <em>"generative"</em> part of the name), and they do this by setting up a competition (hence <em>"adversarial"</em>).
    </p>

    <h2>What's happening in the visualization?</h2>
    <p>
      GANs are complicated beasts, and the visualization has a lot going on. Here are the basic ideas.
    </p>
    
    <p>
      First, we're not visualizing anything as complex as generating realistic images. Instead, we're showing a GAN that learns a distribution of points in just <em>two dimensions</em>. There's no real application of something this simple, but it's much easier to show the system's mechanics. For one thing, probability distributions in plain old 2D (x,y) space are much easier to visualize than distributions in the space of high-resolution images.  
    </p>

    <h4>
      Pick a data distribution.
    </h4>
    
    <p>
      At top, you can choose a probability distribution for GAN to learn, which we visualize as <span style="color: var(--real-data-color);">a set of data samples</span>. Once you choose one, we show them at two places: a smaller version in the <em>model overview graph</em> view on the left; and a larger version in the <em>layered distributions</em> view on the right.
    </p>
    
    <div class="supporting-figure">
      <img src="figures/figure-data-selected.png" />
      <div class="figure-caption">
        Figure 1. Selected data distribution is shown at two places.
      </div>
    </div>
    
    <p>
      We designed the two views to help you better understand how a GAN works to generate realistic samples:
      <br />
      (1) The <strong>model overview graph</strong> shows the architecture of a GAN, its major components and how they are connected, and also visualizes results produced by the components;
      <br />
      (2) The <strong>layered distributions</strong> view overlays the visualizations of the components from the model overview graph, so you can more easily compare the component outputs when analyzing the model.
    </p>
    
    <h4>
      Let training begin.
    </h4>
    
    <p>
      To start training the GAN model, click the play button (<img class="icon" src="figures/figure-icon-play.png" />) on the toolbar. Besides <span style="color: var(--real-data-color);">real samples</span>  from your chosen distribution, you'll also see <span style="color: var(--generator-color);">fake samples</span> that are generated by the model. Fake samples' positions continually updated as the training progresses. A perfect GAN will create fake samples whose distribution is indistinguishable from that of the real samples. When that happens, in the layered distributions view, you will see the two distributions nicely overlap.
    </p>
    
    <div class="supporting-figure">
      <img src="figures/figure-animated-ring.gif" />
      <div class="figure-caption">
        Figure 2. Fake samples' positions continually updated as the training progresses. Then, the distributions of the real and fake samples nicely overlap.
      </div>
    </div>
    
    <h4>
      Visualizing generator and discriminator.
    </h4>
    
    <p>
      Recall that the <span style="color: var(--generator-color);">generator</span> and <span style="color: var(--discriminator-color);">discriminator</span> within a GAN is having a little contest, competing against each other, iteratively updating the <span style="color: var(--generator-color);">fake samples</span> to become more similar to the <span style="color: var(--real-data-color);">real ones</span>. GAN Lab visualizes the interactions between them. 
    </p>
    
    <p>
      <strong>Generator.</strong>
      As described earlier, the <span style="color: var(--generator-color);">generator</span> is a function that transforms a random input into a synthetic output. In GAN Lab, a random input is a 2D sample with a (x, y) value (drawn from a uniform or Gaussian distribution), and the output is also a 2D sample, but mapped into a different position, which is a <span style="color: var(--generator-color);">fake sample</span>. One way to visualize this mapping is using <em>manifold</em> <small><a href="http://colah.github.io/posts/2014-03-NN-Manifolds-Topology/">[Olah, 2014]</a></small>. The input space is represented as a uniform square grid. As the function maps positions in the input space into new positions, if we visualize the output, the whole grid, now consisting of irregular quadrangles, would look like a warped version of the original regular grid. The area (or density) of each (warped) cell has now changed, and we encode the density as opacity, so a higher opacity means more samples in smaller space. A very fine-grained manifold will look almost the same as the visualization of the fake samples. This visualization shows how the generator learns a mapping function to make its output look similar to the distribution of the <span style="color: var(--real-data-color);">real samples</span>.
    </p>
    
    <div class="supporting-figure">
      <img src="figures/figure-generator-transformation.png" />
      <div class="figure-caption">
        Figure 3. The generator's data transformation is visualized as a manifold, which turns input noise (leftmost) into fake samples (rightmost).
      </div>
    </div>
    
    <p>
      <strong>Discriminator.</strong>
      As the generator creates <span style="color: var(--generator-color);">fake samples</span>, the <span style="color: var(--discriminator-color);">discriminator</span>, a binary classifier, tries to tell them apart from the <span style="color: var(--real-data-color);">real samples</span>. GAN Lab visualizes its decision boundary as a 2D heatmap (similar to <a href="http://playground.tensorflow.org"><em>TensorFlow Playground</em></a>). The background colors of a grid cell encode the confidence values of the classifier's results. <span style="background-color: var(--real-data-color); color: white;">Darker green</span> means that samples in that region are more likely to be real; <span style="background-color: var(--generator-color); color: white;">darker purple</span>, more likely to be fake. As a GAN approaches the optimum, the whole heatmap will become more <span style="background-color: rgb(117, 117, 117); color: white;">gray</span> overall, signalling that the discriminator can no longer easily distinguish fake examples from the real ones.
    </p>
    
    <div class="supporting-figure">
      <img src="figures/figure-discriminator-heatmap.png" />
      <div class="figure-caption">
        Figure 4. The discriminator's performance can be interpreted through a 2D heatmap. Here, the discriminator is performing well, since most real samples lies on its classification surface’s green region (and fake samples on purple region).
      </div>
    </div>
    
    <h4>
      Understanding interplay between generator and discriminator.
    </h4>
    
    <p>
      In a GAN, its two networks influence each other as they iteratively update themselves. A great use for GAN Lab is to use its visualization to learn how the <span style="color: var(--generator-color);">generator</span> incrementally updates to improve itself to generate fake samples that are increasingly more realistic. The generator does it by trying to fool the <span style="color: var(--discriminator-color);">discriminator</span>. The generator's loss value decreases when the discriminator classifies <span style="color: var(--generator-color);">fake samples</span> as <span style="color: var(--real-data-color);">real</span> (bad for discriminator, but good for generator). GAN Lab visualizes gradients (as <span style="color: var(--gradient-color);">pink lines</span>) for the fake samples such that the generator would achieve its success.
    </p>

    <div class="supporting-figure">
      <img src="figures/figure-gradients-interplay.png" />
      <div class="figure-caption">
        Figure 5. Fake samples' movement directions are indicated by the generator’s gradients (pink lines) based on those samples' current locations and the discriminator's curren classification surface (visualized by background colors).
      </div>
    </div>
    
    <p>
      This way, the <span style="color: var(--generator-color);">generator</span> gradually improves to produce samples that are even more realistic. Once the fake samples are updated, the <span style="color: var(--discriminator-color);">discriminator</span> will update accordingly to finetune its decision boundary, and awaits the next batch of fake samples that try to fool itself. This iterative update process continues until the discriminator cannot tell <span style="color: var(--real-data-color);">real</span> and <span style="color: var(--generator-color);">fake</span> samples apart.
    </p>
    
    <h4>
      Playing with interactive features.
    </h4>
    
    <p>
      GAN Lab has many cool features that support interactive experimentation.
      <dl>
        <dt>
          Interactive hyperparameter adjustment
        </dt>
        <dd>
          Click the edit icon (<img class="icon" src="figures/figure-icon-edit.png" />) to reveal individual hyperparameters, and edit them on the fly during training.
        </dd>
        <dt>
          User-defined data distribution
        </dt>
        <dd>
          If you don't like our selection of distributions, draw your own by clicking the icon (<img class="icon" src="figures/figure-icon-draw.png" />) at the end of the data distributions list.
        </dd>
        <dt>
          Slow-motion mode
        </dt>
        <dd>
          Lost track of the animation? Then you can slow it down by clicking the slow-motion icon (<img class="icon" src="figures/figure-icon-slow.png" />) to enter slow-mo. 
          <a class="video-part-link" data-start="111" data-end="138">Check out this video</a>
        </dd>
        <dt>
          Manual step-by-step execution
        </dt>
        <dd>
          If you want more control, you can manually train individual iteration step by step by clicking the icon (<img class="icon" src="figures/figure-icon-step.png" />).
          <a class="video-part-link" data-start="139" data-end="190">Check out this video</a>
        </dd>
      </dl>
    </p>
    
    <p>
      Check out the following video for a quick look at GAN Lab's features.
      <ul>
        <li class="video-part-link" data-start="1" data-end="37">
          Introduction of GAN Lab 
          <small>(0:00-0:38)</small>
        </li>
        <li class="video-part-link" data-start="38" data-end="64">
          Training of a simple distribution with hyperparameter adjustments 
          <small>(0:38-1:05)</small>
        </li>
        <li class="video-part-link" data-start="65" data-end="110">
          Training of a user-defined distribution
          <small>(1:05-1:51)</small>
        </li>
        <li class="video-part-link" data-start="111" data-end="138">
          Slow-motion mode 
          <small>(1:51-2:19)</small>
        </li>
        <li class="video-part-link" data-start="139" data-end="190">
          Manual step-by-step execution 
          <small>(2:19-3:10)</small>
        </li>
      </ul>

      <iframe id="video-demo-iframe" width="640" height="302" 
        style="width: 640px;"
        src="https://www.youtube.com/embed/eTq9T_sPTYQ?rel=0&amp;enablejsapi=1"
        frameborder="0" allowfullscreen>
      </iframe>
    </p>

    <h2>How is it implemented?</h2>
    <p>
      GAN Lab uses <a href="https://js.tensorflow.org/"><em>TensorFlow.js</em></a>, 
      an in-browser GPU-accelerated deep learning library. 
      Everything, from model training to visualization, is implemented with 
      JavaScript. You only need a web browser like Chrome to run GAN Lab.
      Our implementation approach significantly broadens people's access to 
      interactive tools for deep learning.
      The source code is available on 
      <a href="https://github.com/poloclub/ganlab">GitHub</a>.
    </p>

    <h2>Who developed GAN Lab?</h2>
    <p>
      GAN Lab was created by 
      <a href="http://minsuk.com">Minsuk Kahng</a>,
      <a href="https://twitter.com/nsthorat">Nikhil Thorat</a>, 
      <a href="https://www.cc.gatech.edu/~dchau/">Polo Chau</a>, 
      <a href="http://www.fernandaviegas.com/">Fernanda Viégas</a>, and 
      <a href="http://www.bewitched.com/">Martin Wattenberg</a>,
      which was the result of a research collaboration between 
      Georgia Tech and Google
      Brain/<a href="https://ai.google/research/teams/brain/pair">PAIR</a>.
      We also thank Shan Carter and Daniel Smilkov, 
      <a href="https://research.google.com/bigpicture/">
        Google Big Picture team</a> and 
      <a href="https://ai.google/research/teams/brain/pair">
        Google People + AI Research (PAIR)</a>, and 
      <a href="http://vis.gatech.edu/">Georgia Tech Visualization Lab</a> 
      for their feedback.
    </p>
    
    <p style="margin-bottom: 0;">
      For more information, check out 
      <a href="http://minsuk.com/research/papers/kahng-ganlab-vast2018.pdf">
        our research paper</a>:     
    </p>

    <a href="http://minsuk.com/research/papers/kahng-ganlab-vast2018.pdf">
      <img src="figures/figure-paper-screenshot.png" style="float: left;" />
    </a>

    <div class="citation">      
      <a href="http://minsuk.com">Minsuk Kahng</a>,
      <a href="https://twitter.com/nsthorat">Nikhil Thorat</a>, 
      <a href="https://www.cc.gatech.edu/~dchau/">Polo Chau</a>, 
      <a href="http://www.fernandaviegas.com/">Fernanda Viégas</a>, and 
      <a href="http://www.bewitched.com/">Martin Wattenberg</a>.
      "GAN Lab: Understanding Complex Deep Generative Models using 
      Interactive Visual Experimentation."
      <i>IEEE Transactions on Visualization and Computer Graphics, 25(1) 
        (<a href="http://ieeevis.org/year/2018/welcome">VAST 2018</a>)</i>, 
      Jan. 2019.
    </div>
  </div>

  <script type="text/javascript">
    const tag = document.createElement('script');
    tag.id = 'iframe-demo';
    tag.src = 'https://www.youtube.com/iframe_api';
    const firstScriptTag = document.getElementsByTagName('script')[0];
    firstScriptTag.parentNode.insertBefore(tag, firstScriptTag);
  
    let player;
    function onYouTubeIframeAPIReady() {
      player = new YT.Player('video-demo-iframe', {
        events: {
          'onReady': onPlayerReady
        }
      });
    }
    function onPlayerReady(event) {
      player.mute()
    }
    
    const videoLinks = document.querySelectorAll('.video-part-link');
    Array.from(videoLinks).forEach((linkElement) => {
      linkElement.addEventListener('click', () => {
        const startSecond = linkElement.getAttribute('data-start');
        player.seekTo(startSecond);
        player.playVideo();
      });
    });
  </script>
</body>

</html>
